Typically, a picture retrieving apparatus and a picture retrieving method using a feature of a picture have been traditionally used to retrieve a similar picture from accumulated pictures.
This kind of a conventional picture retrieving technique employs a method of calculating a color histogram information contained in a picture and then retrieving a similar picture based on its similarity. Here, the color histogram information implies a picture feature amount composed of representative colors indicative of the pictures and its configuration rate. As the color histogram information, it is possible to use a histogram with regard to all colors contained in the picture, a histogram composed of one or a plurality of representative colors and its color configuration rate, and the like.
On the contrary, an example of a picture retrieving apparatus using an orthogonal conversion coefficient of the picture as the picture feature amount is disclosed in Japanese Laid Open Patent Application (Japanese Patent Application Number; Heisei, 11-059432) filed by the same applicant. Here, the orthogonal conversion coefficient of the picture is composed of a value indicative of an energy for each band, which is obtained by dividing a frequency distribution of a spectrum of a color (average color) of the picture into a plurality of bands, and carrying out a frequency analysis by using an orthogonal matrix.
FIG. 9 is a block diagram showing the configuration of the main portion of this conventional picture retrieving apparatus, and shows the picture retrieving apparatus including: a picture feature amount generator 102 for generating a feature amount 103 from a picture data 101; a feature amount memory 104 for storing a feature amount in advance; and a similarity calculator 106 for calculating a similarity 107 between the feature amount 103 and a feature amount 105 contained in the feature amount memory 104.
Here, the picture feature amount generator 102 is composed of: a contraction picture generator 110 for generating a contraction picture 111 from the picture data 101; a frequency analyzer 112 for performing a frequency analysis on the contraction picture 111; and a direct current component and partially alternating current component extractor 114 for extracting as the picture feature amount 103 a direct current component and a partially alternating component from the orthogonal conversion coefficient 113 obtained from the frequency analyzer 112.
Also, an example of the conventional picture retrieving apparatus in which the picture feature amounts are mixed is disclosed in Japanese Laid Open Patent Application (JP-A-Heisei, 09-237343).
FIG. 10 is a block diagram showing the configuration of the main portion of this conventional picture retrieving apparatus. It includes: an input unit 201 for inputting a picture feature amount; a picture feature amount memory 202 for storing the picture feature amount; and a similarity calculator 203 for calculating a similarity between the picture feature amount inputted from the input unit 201 and the picture feature amount stored in the picture feature amount memory 202.
The picture feature amount memory 202 is composed of an orthogonal conversion coefficient memory 204 for storing an orthogonal conversion coefficient of the picture; and a color histogram memory 205 for storing a color histogram information. The similarity calculator 203 is composed of a picture feature amount judging unit 206, a color histogram similarity calculator 207 and an orthogonal conversion coefficient similarity calculator 208.
The picture feature amount judging unit 206 judges whether a picture feature amount 209 inputted from the input unit 201 is the color histogram information or the orthogonal conversion coefficient. If it is judged as the color histogram information by the picture feature amount judging unit 206, the color histogram similarity calculator 207 calculates a similarity 212 between a picture feature amount 210 composed of a color histogram information outputted by the picture feature amount judging unit 206 and a color histogram information 211 read from the color histogram memory 205. If it is judged as the orthogonal conversion coefficient by the picture feature amount judging unit 206, the orthogonal conversion coefficient similarity calculator 208 calculates a similarity 215 between a picture feature amount 213 composed of an orthogonal conversion coefficient outputted by the picture feature amount judging unit 206 and an orthogonal conversion coefficient 214 read from the orthogonal conversion coefficient memory 204.
However, the above-mentioned apparatuses in the respective conventional examples have the following various problems.
As a first problem, the conventional apparatus can not carry out a comparative retrieval between different kinds of picture feature amounts. For example, if a picture feature amount of a query picture is the orthogonal conversion coefficient, the retrieval can not be performed on a database including a color histogram information of a retrieval target picture as the feature amount.
As a second problem, in order to attain the retrieval corresponding to the plurality of kinds of picture feature amounts, the similarity calculators corresponding to the respective picture feature amounts are required to thereby result in a large size of the configuration of the retrieving system. Namely, if the plurality of kinds of picture feature amounts are included, it is necessary to jointly have the similarity calculators corresponding to all the kinds of the picture feature amounts. So, the size of the similarity calculator is made larger. Also, if the plurality of kinds of similarity calculators are included, the picture feature amounts are required for all the similarity calculators.
As a third problem, a raw picture is required when the different kinds of the picture feature amounts are generated. This is because the picture feature amount is extracted from the raw picture.
Japanese Laid Open Patent Application (JP-A-Heisei, 10-301948) discloses the following method of retrieving a picture and an image. It investigates a spatial distribution of a color in an exemplification picture proposed as a retrieval key, and thereby divides the exemplification picture into a plurality of parts (1). It divides the exemplification picture and a retrieval target picture into a plurality of roughness as blocks, and calculates a color histogram in each block (2). It compares the part with the color histogram in the block, for the exemplification picture and the retrieval target picture, and extracts the block constituting the part, and then generates a part picture indicative of a region constituting the part after a hierarchical lamination (3). Next, it calculates an adjacent histogram in which a position relation between the parts is represented as a histogram, by using this part picture (4). Finally, it compares the adjacent histogram obtained from the exemplification picture with the adjacent histogram obtained from the retrieval target picture, and accordingly judges the presence or absence of a subject in the retrieval target picture (5).
Japanese Laid Open Patent Application (JP-A-Heisei, 11-102439) discloses the following method of retrieving a subject in a picture. At first, it investigates a spatial distribution of a color in a retrieval key picture in the subject, and divides into a plurality of parts (1). It divides the retrieval key picture and a retrieval target picture into blocks, and calculates a color histogram in each block (2). It generates a part picture indicating the sureness that each point in the retrieval key picture and the retrieval target picture belongs each part, by using a color histogram matching in which a similarity of a color is reflected (3). It calculates an adjacent histogram representative of a position relation between the parts, by using the part picture (4). Next, it updates a value of the sureness with regard to the part picture generated from the retrieval target picture by using this adjacent histogram, and suppresses an influence on background (5). Finally, it compares the adjacent histograms calculated from the part pictures of the updated retrieval target picture and the retrieval key picture with each other (6), and accordingly judges the presence or absence of the subject in the retrieval target picture (7).
Japanese Laid Open Patent Application (JP-A-Heisei, 11-316819) discloses the following picture retrieving apparatus. With regard to each of a plurality of picture data, a picture feature amount extractor 14 divides a picture into a plurality of blocks, and calculates a feature amount for each block. A feature amount labeling unit 15 gives a label on the basis of the feature amount of each block, and generates a label histogram information indicative of a histogram of the given label, and also records in a picture management database 18, and then records in a component index 19 from which a picture IG of the picture including the label and the included number can be retrieved with each label as a key A pattern matching unit 16 establishes a retrieval target label to be defined as a retrieval condition and its range of the included number, and refers to the component index 19 under the established retrieval condition, and then extracts the picture.
By the way, as the U.S. patents in relation to this application, there are U.S. Pat. No. 4,646,250 February, 1987 Childress 395/149, U.S. Pat. No. 4,672,683 June, 1987 Matsueda 382,305, U.S. Pat. No. 4,716,404 December, 1987 Tabata et al. 395/164, U.S. Pat. No. 4,829,453 May, 1989 Katusta et al. 382/305, U.S. Pat. No. 4,944,023 July, 1990 Imao et al. 382/240, U.S. Pat. No. 5,012,334 April, 1991 Etra 348/107, U.S. Pat. No. 5,148,522 September, 1992 Okazaki 395/161, U.S. Pat. No. 5,179,652 January, 1993 Rozmanith et al. 395/155, U.S. Pat. No. 5,202,828 April, 1993 Vertelney et al. 395/159, U.S. Pat. No. 5,220,648 June, 1993 Sato 395/155, U.S. Pat. No. 5,381,158 January, 1995 Takahara et al. 395/161, U.S. Pat. No. 5,421,008 May, 1995 Banning et al. 395/600, U.S. Pat. No. 5,428,727 June, 1995 Kurosu et al. 395/155, U.S. Pat. No. 5,434,966 July, 1995 Nakazawa at al. 395/161. U.S. Pat. No. 5,579,471 November, 1996 Barber et al. 395/326, U.S. Pat. No. 5,647,058 July, 1997 Agrawal et al. 395/326, U.S. Pat. No. 5,852,823 December, 1998 De Bonet 707/6, U.S. Pat. No. 5,893,095 April, 1999 Jain et al. 707/6, and U.S. Pat. No. 6,269,358 July, 2001 Hirata.
The present invention is accomplished in view of the above mentioned problems. Therefore, a first object of the present invention is to provide a picture retrieving apparatus and a picture retrieving method that can carry out a retrieval, whether a picture feature amount is an orthogonal conversion coefficient of a picture or a color histogram information, in a system in which a picture retrieval using the color histogram information of the picture and a picture retrieval using the orthogonal conversion coefficient of the picture are mixed, and a recording medium for recording a retrieval program.
A second object of the present invention is to provide a picture retrieving apparatus and a picture retrieving method that can simplify the system configuration without any large loss of a retrieving performance, in a system in which a picture retrieval using a color histogram information of a picture and a picture retrieval using an orthogonal conversion coefficient of the picture are mixed, and a recording medium for recording a retrieval program.